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ABSTRACT ^.^ 

The main purposes of this study were to identify the 
evaluation needs of students, teachers, and principals, and to 
develop recommendations for an evaluation system within the school 
building. Students, teachers, and principals were surveyed to depict 
the availability and importance of eight different categories of 
evaluative information derived from theoretical framework suggested 
by stufflebeam (The CIPP Model) and by Scriven (formative and 
summative evaluation) . The study showed that while information on 
outcomes is the most available evaluative information in school, 
school people show a great concern for other kinds of evaluative 
information. Students and principals showed preference for formative 
rather than summative evaluation. (Author) 
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INTRODUCTION 

School staffs have usually evaluated their work by testing their 
students. As argued in the evaluation literature, test scores, while 
important, are insufficient to fully serve evaluation purposes. To 
Identify one basis for expanding the scope of school -based evaluation, 
this study asked students, teachers, and principals to assign priorities 
to alternative information items that are theoretically important for 
evaluating schooling. 

The initial lists of potential information needs were derived 
from a matrix that combines evaluation concepts suggested by Scriven^ 
and Stufflebeam^* The row headings of the matrix (see Figure 1) are 
Scriven's well-known concepts of formative and summative evaluation— 
which are purposes to be served by evaluation findings. The column 
headings are Stufflebeam's concepts of context, input, process, and 
product evaluation— which denote different types of information* This 
2x4 matrix suggests eight types of evaluative information that are po- 
tentially important in assessing school programs. 

The first is formative-context evaluation . It involves needs 
assessment; but it also searches for opportunities, such as advances 
in technology and special funding sources, that are potentially availa- 
ble for meeting the needs. Finally, it diagnoses problems that must 

^Michael Scriven, "The Methodology of Evaluation" in Robert E. 
Stake (Ed.), Perspectives of Curriculum Evaluation, AERA Monograph 
Series on Evaluation, No. 1 (Chicago: Rand McNally, 1967). 

^Daniel L. Stufflebeam, et al.. Educational Evaluation and De- 
cision Making (Itasca, 111.: Peacock Publishers, 1971). 
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Figure 1 : Eight Categories of Evaluative Information 



be solved before the opportunities can be used to serve the needs. 
The main use of formative-context evaluation is to assist in the for- 
mulation of goals and objectives. 

A secoiid type of information is summative-context evaluation . It 
assesses the merit of goals that were chosen as compared with other 
possibilities. Key criteria concern the goals' situational relevance, 
adherence to democratic ideals, and clarity. A main use of summative- 
context evaluation is to aid persons or groups to be accountable to 
their publics, but it also aids these publics to draw conclusions 
about whether an effort was well-intentioned. 

Formative-input evaluation , the third kind of information, iden- 
tifies and assesses the potential costs, benefits, and feasibility of 
alternative plans for achieving specified objectives. The main function 
of formative-input evaluation is to assist in the development and 
adoption of cost/ effective plans. 



The fourth kind of information conies from summative-lnput evalu- 
ation . It identifies and judges a plan that was previously chosen, 
especially in comparison to other possibilities. Sunwattve-input 
evaluation aids persons or groups to defend their past choices of plans, 
and it provides interested audiences with a basis for judging whether 
the choice of a particular plan was warranted. 

The fifth kind'of information is formative-process evaluation . 
It provides continual feedback about how well a plan is being iir^ple- 
mented. This feedback concarns limitations in both the plan under 
operating conditions, and its execution. The purpose of formative- 
process evaluation is to aid persons to carry out their plans. 

Summative process-evaluation denotes the sixth kind of information. 
It involves retrospective descriptions and judgments of the actual 
process that was implemented in a. completed enterprise. This infor- 
mation assists persons to defend their past actions in carrying out 
their responsibilities; and it assists them and their audiences to 
determine whether the conceptual plan had a complete and fair operational 
trial. It also helps them to interpret outcomes. 

Formative-product evaluation > the seventh kind of information, 
provides continual feedback about results. Are needs being met? Are 
the opportunities being used? Are problems being solved? Are objec- 
tives being achieved? Answers to these questions aid persons to recycle 
their activities toward the end of continually obtaining a better effect. 

The eighth and final kind of information denoted by our matrix 
is sumnative-product evaluation . It describes, interprets, and judges 
the end results of a class, school year, etc. This information is the 



ultimate basis for accountability and conclusions related to the success 
of an effort. 

We believe these categories of evaluation provide a comprehensive 
view of the information that might be included in school evaluation 
programs. However, there is very little evidence to suggest which of 
the kinds of information should receive the highest priority in efforts 
to expand school evaluation work. 

We investigated this issue in terms of two main questions: 

1) What information is presently available to school personnel? 

2) What information do they perceive to be most important? 

STUDY PROCEDURES 

To address the questions of this stucly, we surveyed students, 

teachers, and principals, using questionnaires designed to depict 

their perceptions of the availability and importance of different kinds 

of evaluative information.^ For feasibility reasons the survey was 

limited to the high schools of Michigan, and the study has not been 

replicated. For these and related reasons, our results must be con- 

we hope 

sidered tentative, and^they will lead to further related research. 

We drew three samples. The first included 185 high school teachers 
who were randomly drawn from the Michigan Professional Personnel Regis- 
ter, which is prepared annually by the Michigan State Department of 



^For a complete description of the study procedures and an 
itemized presentation of its findings see: David Nevo, "Evalu- 
ation Priorities of Students, Teachers, and Principals." Ph. D. 
Dissertation, The Ohio State University, 1974. 



Education and Itsts tnfonnatton concerning all professtonal personnel 
in Mtchigan*s public schools. The second sample included 164 high 
school principals randomly drciwn from a list of all Michigan public 
high school principals provided in the 1973-74 Michigan Eclucational 
Directory. Thus, representative samples were obtained for teachers 
and their principals. However, no representative sample was obtained 
for students, since this sample was constrained by our ability to ob- 
tain the cooperation of schools in administering questionnaires to their 
students. Three local school districts agreed to participate in the 
study, providing a total of five high schools. A random sample of 
100 students was drawn from a group of 228 students in the tenth and 
eleventh grades who returned usable questionnaires in those five schools. 

Three parallel questionnaires— for students, teachers, and prin- 
cipals—were the main measurement tools of this study* Each question- 
naire contained forty items, including five for each of eight categories 
of evaluative information (context, input, process, and product infor- 
mation, as differentiated by formative and summative purposes). 

The questionnaires were developed according to the fo** lowing 
procedures: 

(a) A pool of 214 items was developed describing information on 
the context, input, process^ and product of educational activities that 
would be intended to serve either formative or summative purposes. 
The pool included 74 items for students; 69 for teachers; ana 71 for 
principals. 

Cb) Eight judges critiqued the initial items and classified them 
into the study's eight information categories. 



Cc) Based on the judges' classiftcatton, five items were chosen 
for each of the eight categories of evaluative information, and this 
was done for each of the three instruments. 

Cd) For each instrument the forty selected items were arranged 
in random sequence, and the ftnal instruments were prepared, such that 
respondents would rate each item, according to a five-point scale, for 
both availability and importance, with 1 as low (availability or im- 
portance) and 5 as high. 

Once prepared, the questionnaires were administered to students 
tn their classrooms; while the 185 teachers and 164 principals received 
their questionnaires by mail. A follow-up mailing to non-respondents 
was conducted two weeks after the first mailing. Ultimately 92 (50%) 
of the teachers and 85 (52%) of the principals returned their question- 
naires. 

Separate analyses were done for availability and importance, and 
for each study group. Within each analysis, mean scores were obtained 
for each information category in our theoretical matrix, based on the 
scores obtained for the items included in each category. These means 
could range from 1 to 5, referring tc the following descriptive scale 
applied separately for availabil ity and importance: 

1 - never available or of no importance 

2 - rarely available or of minimal importance 

3 - sometimes available or of moderate importance 

4 -.frequently available or of high importance 

5.- very frequently available or of very high importance 

The differences among means were assessed by a two-factor analysis 
of variance with repeated measures. Significant F ratios were further 
investigated with Tukey's Honestly Significant Difference (HSD) procedure. 
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FINDINGS 



Overall, the subjects in all three groups Indicated th.at all types 
of information were "rarely" or "sometimes" available to them* as can 
be seen from the cell means in Table 1 ranging from 2.2 to 3.1 on our 
five-point scale. Little difference among groups was observed, as 
noted by the gross averages ^ the mean scores: 2.5 for students, 
2.4 for teachers, and 2.7 for principals. 




Table 1 



Mean Scores for Availability of Evaluative 
Information in Three Study Groups 



Study Group 


HO 1 e OT 




Type of Evaluative 


Information 


Evaluation 


Context 


Input- 


Process 


Product 


All Types 




Formative 


2.9 


2.6 


2.3 


2.6 


2.6 


Students 


Summatlve 


2.4 


2.4 


2.2 


2.6 


2.4 




Both Roles 


2.7 


2.5 


2.3 


2.6 


2.5 




Formative 


2.6 


2.2 


2.3 


2.5 


2.4 


Teachers 


Summative 


2.5 


2.2 


2.3 


2.8 


2.4 




Both Roles 


2.6 


2.2 


2.3 


2.7 


2.4 




Formative 


2.8 


2.5 


2.6 


3.0 


2.7 


Principals 


Summative 


2.7 


2.5 


2.6 


3.1 


2.7 




Both Roles 


2.8 


2.5 


2.6 


3.0 


2.7 
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However, analyses of variance revealed that all three groups 
perceived that some types of information are more available than others. 
Sig;iificant differences at the .01 level were found for all three 
groups as regards their perception of the availability of context, 
input, process, and product information. Also a significant F ratio 
indicated that the student group rated formative evaluation as more 
available than summative evaluation. But, superseding these findings 
were interaction effects for all three groups that were significant 
at the .01 level. The results from the Tukey tests for all three 
samples suggested that product and context evaluation information are 
the most available within the school. 

As regards importance, all three groups viewed most of the 
information items, that might be made available to them, to be of 
"high importance." Students and principals rated the importance of 
information with an average ratine, of 3.7, and teachers followed with 
an average rating of 3.6 on our five-point scale (see Table 2). We 
infer that the three study groups believe that, while much of the 
information we described is relatively unavailable to them, it would 
be of much importance to them if they could get it. 

Our further analyses of the ratings for all three groups 
revealed significant differences concerning the ratings of importance 
that were assigned to the eight categories of information. The.F 
ratios for all three groups were significant at the .01 level as 
regards variances among the rated importance of context, input, 
process, and product information. For the student group and the 
principal group, the F ratio for the 'ariance between formative 
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Table 2 

Mean Scores for Importance of Evaluative 
Information in Three Study Groups 



Study Group 


Role of 


Type of Evaluative Information 


Evaluation 


Context 


Input 


Process 


Product 


All Types 




Formative 


3.8 


3.9 


3.5 


3.9 


3.8 


Students 


Summative 


3.8 


3,5 


3.3 


3,7 


3.6 




Both Roles 


3.8 


3.7 


3.4 


3.8 


3.7 

if 




Formative 


3.9 


3.4 


3.4 


3.6 


3.6 


Teachers 


Summative 


3.8 


3.5 


3,4 


3.6 


3.6 




Both Roles 


■ 3.9 


3.4 


3.4 


3.6 


3.6 




Formative 


3.9 


3.7 


3.7 


3.8 


3.8 


Principals 


Summative 


3.8 


3.4 


3.7 


3.9 


3.7 




Both Roles 


3.8 


3.5 


3.7 


3.9 


3.7 



and summative evaluation was also significant. Significant (.01 level) 
interaction effects were also found for the student and principal data. 

Thus, the patterns of difference in ratings of importance must 
be considered differentially for the three groups. The data from 
teachers suggest that they perceive context and product evaluation 
to be more important than process and input evaluation with no 
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preference for summative or formative evaluation. Principals showed 
a statistically significant preference for formative-context 
evaluation and summative-product evaluation. Students showed a general 
preference for formative evaluation especially in regard to input and 
product evaluation. 

SUMMARY 

Overall, students, teachers, and principals perceive that 
evaluative information of the type described in our study is only 
occasionally available; and they perceive that context and product 
infomation is the most available. Compared to their low rankings 
for availability, they indicate that, overall, the information items 
considered in this study are of much importance to them. Teachers 
especially desire to have both formative and summative context and 
prciuct information. Students especially want formative -input and 
formative-product information. And principals express a special need 
for formative-context and summative-product information. 

Although the study's findings should be considered tentative 
and interpreted with caution, they might have significant implications 
for the development of an evaluation system within the school. The 
instruments devised in this study should be of use in replicating this 
study and in conducting further related research. Moreover, the 
instruments are potentially useful for school staffs that want to assess 
their needs for evaluat-^ve information, and they should provide e 
stimulus and frame of reference for in-service training in evaluation. 
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Overall, the study supports the contention that there is a need to 
expand and improve school -based evaluation programs, so that they will 
be of greater service to students, teachers, principals, and other 
involved personnel. 
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